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Dr.  John  Tangnev 
AFOSR,  Building  410 
Bolling  Airforce  Base 
Washington,  D.C.  20332-6448 

Dear  John, 

t 

Here  it  is  at  last  -  the  proposal  1  promised  to  send  you  6  months  ago!  Following  your 
advice  I  have  kept  it  "conservative."  We  will  also  be  working  on  phantom  contours,  filling 
in  etc.,  but  we  have  left  these  out  in  the  interest  of  brevity. 

The  proposal  deals  mainly  with  the  important  but  neglected  problem  of  visual 
surface  representation  -  especially  the  role  played  by  segmentation  in  this  process.  As  y  ou 
know,  our  laboratory  has  been  involved  in  the  discovery  of  several  novel  perceptual  effects 
such  as  ’motion  capture’  and  stereoscopic  capture.  Our  goal  will  be  to  continue  this 
research  and  to  obtain  more  detailed  parametric  data  on  these  illusions. 

As  >ou  can  see  our  Yvork  is  quite  critical  of  DaY  id  Marr  and  should  therefore  not  be 
reviewed  by  any  of  his  former  disciples  or  associates.  It  should  be  kept  in  mind,  hoYveY  er 
that  although  our  experiments  has  e  called  into  question  Yvhole  classes  of  models,  they  has  e 
also  inspired  several  neYv  computational  models  of  motion  processing  and  stereopsis(e.g.  by 
Bulthoff,  Yuille,  and  others). 


We  Yvould  also  prefer  that  it  was  not  reY  iewed  by  Gibsonians.  On  the  other  hand,  any 
eminent  perceptual  psychologist  would  be  appropriate  as  a  rex-jewer,  e.g.  Triesman,  Julesz, 
Cavanagh,  or  Rock. 

I  enclose  a  summary  of  Yvhat  we  hax  e  accomplished  during  the  last  3  years  that  we 
haxe  recehed  funding  from  your  agency.  We  look  fonvard  to  your  continued  support. 


6-flFoSfi.  Sr1-uH)S 

Li  A  A  a.  J 
x.  HJoio 


With  best  regards. 


V.S.  Ramachandran,  M.D.,  Ph.D. 
Professor  of  Neurosciences 
and  Psychology 
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Our  research  is  concerned  with  the  question  of  how  the  visual  system  creates  3-D 
representations  of  surfaces.  The  last  two  or  three  decades  have  seen  breathtaking  progress  in  the 
three  disciplines  --  cognitive  psychology,  AI  and  visual  neurophysiology  --  but  they  have  been 
pursued  more  or  less  independently.  Our  research  suggests  that  the  time  is  now  ripe  for  forging 
links  between  these  disciplines  for  an  integrated  approach  to  vision. 

'  .We  have  had  two  goals  in  mind: 

1.  To  develop  conceptual  links  between  neurophysiology  and  perception; 

2.  To  develop  specific  tests  for  computational  models  of  human  vision. 

While  it  is  true  that  some  of  our  work  has  called  into  question  whole  classes  of  "models” 
they  have  also  inspired  several  new  computational  models  (e.g.,  models  of  "motion  capture" 
proposed  by  Bulthoff,  Yuille,  Koch  and  others). 

During  the  last  decade  the  work  of  David  Marr  (1981)  has  had  a  tremendous  impact  on 
our  field  —  mostly  positive.  Indeed,  his  work  has  re-  olutionized  the  study  of  vision  in  a  manner 
analogous  to  the  Chomskyan  revolution  in  linguistics.  There  are,  however,  several  major  pitfalls 
associated  with  his  approach.  Consider  the  four  basic  pillars  on  which  Mart's  edifice  rests: 

1.  Complex  information  processing  systems  --  such  as  human  vision  --  can  be 

studied  at  different  "levels"  --  i.e.,  the  abstract  level  of  the  "computational  problem,"  the  level  of 
algorithm  and  the  level  of  hardware  implementation.  Marr  urges  us  not  to  get  "confused" 
between  these  levels  --  they  must  be  kept  quite  separate  if  we  wish  to  avoid  getting  into 
conceptual  muddles.  -  " 

2.  The  most  important  critical  step  in  understanding  any  visual  process  is  to  clearly 
formulate  the  "computational  problem"  preferably  in  formal  mathematical  language. 

3.  "Segmentation"  of  the  visual  image  is  a  complex,  ill  posed,  and  largely  intractable 
problem  in  AI.  Fortunately,  however,  many  of  the  processes  of  early  vision,  e.g.,  stereopsis, 
motion  correspondence,  structure  from  motion,  shape  from  shading,  etc.,  take  place  prior  to 
image  segmentation.  In  fact  their  output  can  lead  to  segmentation  (e.g.,  Julesz’  stereograms). 
Therefore  we  can  study  and  model  these  processes  without  worrying  too  much  about 
segmentation. 


4.  "Top  down"  processes  based  on  high  level  semantics  have  no  role  in  human 

vision. 


These  four  assumptions  seem  reasonable  enough  at  first  glance,  but  our  work  suggests 
that  none  of  them  really  holds  up  on  careful  scrutiny.  Let  us  consider  each  in  turn. 

1.  The  argument  about  levels  may  be  valid  as  a  logical  point  (one  recalls  Gilbert 
Ryles  remarks  about  "category  mistakes")  but  from  a  strategic  point  of  view  the  advice  is 
misguided.  In  our  view  the  only  sure  way  to  progress  in  understanding  any  biological 
information  processing  system  -  such  as  vision  --  is  to  develop  conceptual  links  between  levels 
instead  of  trying  to  keep  them  separate.  As  an  analogy  consider  the  manner  in  which  our 
understanding  of  the  double  helical  structure  of  the  DNA  molecule  (i.e.,  the  "hardware”) 
completely  transformed  our  understanding  of  classical  heredity  and  genetics  which,  until  then, 
had  remained  a  "black  box"  subject.  There  is  now  a  wealth  of  empirical  evidence  that  the  same 
principle  holds  for  understanding  human  vision,  i.e.,  the  neural  machinery  in  our  visual  pathways 


powerfully  constraints  our  perceptual  experience  of  the  world  (e.g.,  Ramachandran  &  Gregory. 
1978;  Ramachandran,  1987:  Ramachandran,  1991;  Ramachandran  &  Gregory.  1991:  Rogers- 
Ramachandran,  Ramachandran,  1991,  ARVO).  For  example,  we  have  now  devised  a  stimulus 
that  seems  to  selectively  activate  a  "fast”  sign-invariant  contour  processing  system  in  human 
vision  that  might  correspond  to  the  "magno  cellular  pathway"  of  physiologists. 

2.  Understanding  the  "computational  problem"  is  certainly  important,  as  emphasized 
by  Marr,  but  it  is  very  easy  to  prejudge  what  the  problem  actually  is  unless  you  do  experiments, 
e.g.,  what  is  the  computarional  goal  of  color  vision?  Also  "computational  problems"  such  as 
stereo  correspondence,  structure  from  motion  and  the  aperture  problem  were  first  identified  by 
doing  experiments  (e.g.,  by  Julesz,  Wallach  and  others)  and  they  were  not  deduced  from  first 
principles. 

3.  Work  done  in  our  lab  contradicts  Marr’s  claim  that  segmentation  does  not 
influence  early  vision  modules.  What  we  find  in  fact  is  that  image  segmentation  produced  by 
cues  such  as  implied  occlusion,  for  example,  can  powerfully  constrain  the  solution  to  many  early 
vision  problems  such  as  motion  correspondence  (Ramachandran,  1985;  Ramachandran,  1991) 
stereopsis  (Ramachandran,  19S6),  structure  from  motion  (Ramachandran,  Cobb  &  Rogers- 
Ramachandran,  1986);  and  shape  from  shading  (Ramachandran,  1988).  Any  program  of 
research  on  vision  must  take  these  facts  into  account. 

For  example,  we  have  done  several  experiments  which  suggest  that  even  illusory' 
contours  (defined  by  implied  occlusion)  can  profoundly  influence  die  processing  of  stereopsis, 
apparent  motion  and  shape  from  shading. 

4.  The  view  that  "top-down"  processes  play  no  role  in  human  vision  is  contradicted 
by  the  simple  observation  that  hollow  masks  do  not  look  hollow,  they  look  convex.  This  is  true 
even  when  the  visual  system  has  to  override  stereoscopic  disparity  (Helmholtz,  Gregory',  1976) 
or  the  assumption  of  overhead  lighting  (Ramachandran,  1988). 

One  could  argue,  however,  that  this  tendency  has  nothing  to  do  with  familiarity  with 
faces.  The  illusion  may  arise  from  a  generic  assumption  about  the  convexity  of  objects  rather 
than  familiarity  with  faces.  To  test  this,  we  recently  tried  comparing  an  upside-down  hollow 
mask  with  a  hollow  mask  held  upright.  By  walking  away  from  the  mask  until  it  iust  reversed 
(i.e.,  was  seen  as  convex),  we  found  that  larger  disparities  can  be  overridden  by  the  latter  than  by 
the  former.  Since  the  masks  are  otherwise  completely  identical,  the  observed  difference  has  to 
be  a  result  of  the  fact  that  upright  faces  are  more  "face-like"  than  upside-down  ones.  Thus, 
while  it  is  largely  true  that  early  vision  is  relatively  immune  from  semantics  --  our  experiments 
suggest  that  Marr  has  clearly  overstated  his  case. 

In  summary,  our  research  has  called  into  question  several  widely  accepted  dogmas 
concerning  the  mechanisms  of  early  vision.  Also,  we  have  been  able  to  discover  several  novel 
visual  phenomena  (e.g.,  motion  capture,  stereo-capture,  etc.)  and  have  identified  a  wide  range  of 
new  "natural  constraints"  that  govern  the  perception  of  shape-from  shading  (Ramachandran, 
1988;  Kleffner  &  Ramachandran,  1992),  structure  from  motion  and  motion  correspondence. 
Also,  we  have  discovered  striking  perceptual  correlates  of  several  well-known  physiological 
observations  (e.g.,  "phantom  contours"  —  stimuli  which  selectively  activate  the  magnocellular 
pathway;  "filling  in"  of  scotomas  described  by  Gilbert  &.  Wiesel,  Gaines  &  others;  plasticity  of 
cortical  topography,  described  by  Merzenich,  Pons  and  Gatas). 

The  enclosed  reprints  provide  more  detailed  descriptions  of  research  that  we  have  been 
doing  along  these  lines. 


List  of  Publications 

1.  '  Stoner,  G.,  Albright,  T.,  &  Ramachandran,  V.  S.  (1990).  Transparency  and  Coherence  in 

Human  Motion  Perception.  Nature,  344,  153-155. 

2.  Ramachandran,  V.  S.,  &  Anstis,  S.  M.  (1990).  Elusory  displacement  of  equiluminous 
'  kinetic  edges.  Perception,  19,  61 1-616. 

3. -  '  Deutsch,  J.  A.,  Ramachandran,  V.  S.  &  Peli,  E.  (1990).  Binocular  depth  reversals  despite 

familiarity  cues:  An  artifact?  Science,  249,  565-566. 

4.  Nakayama,  K.,  Shimojo,  S.,  &  Ramachandran,  V.  S.  (1990).  Transparency:  Relation  to 
depth,  subjective  contours,  luminance,  and  neon  color  spreading.  Perception,  19,  497- 
513. 

5.  Ramachandran,  V.  S.  (1990).  Visual  percetion  in  people  and  machines.  Aland  the  Eye. 
Sussex,  England:  John  WEey  and  Sons,  21-77. 

6.  Ramachandran,  V.  S.  &  Gregory,  R.  L.  (1991).  Perceptual  filling  in  of  artificially 
induced  scotomas  in  human  vision.  Nature,  350,  699-702. 

7.  Ramachandran,  V.  S.  &  Rogers-Ramachandran,  D.  C.  (1991).  Phantom  contours,  a  new 
class  of  visual  patterns  that  selectively  activates  the  magnocellular  pathway  in  man. 
Bulletin  of  the  Psychonomic  Society,  29,  391-394. 

8.  Ramachandran,  V.  S.  (-1991).  Form,  motion,  and  binocular  rivalry.  Science,  251 ,  950- 
951. 

9.  Kleffner,  D.  &  Ramachandran,  V.  S.  (1992).  On  the  perception  of  shape  from  shading. 
Perception  and  Psychophysics. 

10.  Ramachandran,  V.  S.  (1992).  Visual  perception:  A  biological  perspective.  Neural 
Networks  in  Visual  Processing. 

11.  Ramachandran,  V.  S.  (1992).  2-D  or  not  2-D:  That  is  the  question.  The  Artful  Brain, 
Oxford:  Oxford  University  Press. 

12.  Plummer,  D.  J.,  &  Ramachandran,  V.  S.  (1992).  Perception  of  transparency  in  stationary' 
and  moving  displays,  perception  and  psychophysics.  Perception  and  Psychophysics. 

13.  Ramachandran,  V.  S.  (1992).  Blind  spots.  Scientific  American,  May  1992, 266,  86-91. 

14.  Ramachandran,  V.  S.  (In  press).  Filling  in  gaps  in  perception:  Parti.  Current  directions 
in  psychological  science. 

15. '  Ramachandran,  V.  S.  (In  press).  Filling  in  gaps  in  perception:  Part  II.  Current 

directions  in  psychological  science. 

16.  Ramachandran,  V.  S. ,  Stewart,  M.  &  Rogers-Ramachandran,  D.  (1992).  Perceptual 
correlates  of  massive  conical  reorganization.  Neuro.  Report,  3,  583-586. 


Books 


1.  Appointed  Editor-in-Chief  of  a  four  volume  Encyclopedia  of  human  behavior.  Academic 
Press. 

2.  Two  book  contacts  for  the  Scientific  American  Library  series  -  one  on  the  human  brain 
with  Patricia  Churchland  and  the  other  on  "seeing.” 

Invited.lectures,  appointments,  coiloquiua  etc. 

1.  Invited  to  give  the  presidential  lecture  at  the  Annual  Meeting  of  the  Society  of 
Neuroscience,  Phoenix,  AZ. 

2.  Invited  to  give  a  public  lecture  at  the  250th  anniversary  celebration  of  the  University  of 
Pennsylvania  School  of  Medicine. 

3.  Invited  speaker  at  the  annual  meeting  of  the  Neurosciences  research  program  (NRP)  held 
at  the  Rockefeller  Institute  (March,  1991). 

4.  Invited  "keynote"  speaker  at  the  SPIE  meeting,  San  Diego  (1992). 

5.  Invited  "keynote"  speaker  at  SIGRAPH,  1992,  held  in  Chicago. 

6.  Invited  speaker  at  special  symposium  on  "Neuronal  Group  Selection"  at  the  Rockefeller 
Institute  (NRP),  May,  1992,  organized  by  Max  Cowan  and  Gerald  Edelman. 

7.  Kenneth  Craik  lecture  given  at  Cambridge  University. 

8.  Colloqium  given  at  MIT. 

9.  Colloquium  given  at  Oxford  University. 

10.  Awarded  "Certificate  of  Appreciation  for  Outstanding  Contributions  to  Visual  Science" 
by  the  Optometric  Association  of  America. 

11.  Appointed  McDonnel  Pew  Visiting  Fellow,  Oxford  University,  England. 

12.  Interviewed  on  BBC  television  ("Antenna").  Debate  with  Daniel  C.  Dennett.  Aired  on 
August  8,  1992. 

13.  Interviewed  on  PBS  television  (KCET,  LA)  for  a  program  entitled  "Inside  Information" 
which  aired  nationally. 


